A Multidimensional Approach to Multimodal Dialogue Act Annotation

This paper investigates the benefits of multidimensional approaches to dialogue act annotation, and the advantages of using layered multidimensional ’open’ dialogue act taxonomies. We performed a comparative analysis of a one-dimensional and two multidimensional dialogue act annotation schemes, and concluded that not only does a multidimensional approach support a more accurate analysis of human communication, but contrary to what is often believed it also facilitates dialogue annotation. This conclusion was supported by an analysis of nonverbal expressions in multiparty dialogue. We found that, like verbal ones, nonverbal communicative acts may also serve several communicative functions simultaneously within different dimensions. Finally, we showed that the multiple communicative functions which a dialogue utterance may have, and which form the basis of the multidimensional approach, are automatically learnable in a data-oriented way.